Hand Force Estimation from Acoustic Myography Using Deep Wavelet Scattering Transform and Long Short-Term Memory.

The ability to estimate user intention from surface electromyogram (sEMG) signals is a crucial aspect in the design of powered prosthetics. Recently, researchers have been using regression techniques to connect the user's intent, as expressed through sEMG signals, to the force applied at the fingertips in order to achieve a natural and accurate form of control. However, there are still challenges associated with processing sEMG signals that need to be overcome to allow for widespread and clinical implementation of upper limb prostheses. As a result, alternative modalities functioning as promising control signals have been proposed as source of control input rather than the sEMG, such as Acoustic Myography (AMG). In this study, six high sensitivity array microphones were used to acquire AMG signals, with custom-built 3D printed microphone housing. To tackle the challenge of extracting the relevant information from AMG signals, the Wavelet Scattering Transform (WST) was utilized. alongside a Long Short-Term Memory (LSTM) neural network model for predicting the force from the AMG. The subjects were asked to use a hand dynamometer to measure the changes in force and correlate that to the force predicted by using the AMG features. Seven subjects were recruited for data collection in this study, using hardware designed by the research team. the performance results showed that the WST-LSTM model can be robustly utilized across varying window sizes and testing schemes, to achieve average NRMSE results of approximately 8%. These pioneering results suggest that AMG signals can be utilized to reliably estimate the force levels that the muscles are applying.Clinical Relevance- This research presents a new method for controlling upper limb prostheses using Acoustic Myography (AMG) signals. A novel method mapping the AMG signals to force applied by the corresponding muscles is developed. The presented findings have the potential to lead to the development of more natural and accurate control of human-machine interfaces.

Deep Scattering Transform with Attention Mechanisms Improves EMG-based Hand Gesture Recognition.

Electromyogram (EMG) signals provide valuable insights into the muscles' activities supporting the different hand movements, but their analysis can be challenging due to their stochastic nature, noise, and non-stationary variations in the signal. We are pioneering the use of a unique combination of wavelet scattering transform (WST) and attention mechanisms adopted from recent sequence modelling developments of deep neural networks for the classification of EMG patterns. Our approach utilizes WST, which decomposes the signal into different frequency components, and then applies a non-linear operation to the wavelet coefficients to create a more robust representation of the extracted features. This is coupled with different variations of attention mechanisms, typically employed to focus on the most important parts of the input data by considering weighted combinations of all input vectors. By applying this technique to EMG signals, we hypothesized that improvement in the classification accuracy could be achieved by focusing on the correlation between the different muscles' activation states associated with the different hand movements. To validate the proposed hypothesis, the study was conducted using three commonly used EMG datasets collected from various environments based on laboratory and wearable devices. This approach shows significant improvement in myoelectric pattern recognition (PR) compared to other methods, with average accuracies of up to 98%.

A spatial-temporal linear feature learning algorithm for P300-based brain-computer interfaces.

Speller brain-computer interface (BCI) systems can help neuromuscular disorders patients write their thoughts by using the electroencephalogram (EEG) signals by just focusing on the speller tasks. For practical speller-based BCI systems, the P300 event-related brain potential is measured by using the EEG signal. In this paper, we design a robust machine-learning algorithm for P300 target detection. The novel spatial-temporal linear feature learning (STLFL) algorithm is proposed to extract high-level P300 features. The STLFL method is a modified linear discriminant analysis technique focusing on the spatial-temporal aspects of information extraction. A new P300 detection structure is then proposed based on the combination of the novel STLFL feature extraction and discriminative restricted Boltzmann machine (DRBM) for the classification approach (STLFL + DRBM). The effectiveness of the proposed technique is evaluated using two state-of-the-art P300 BCI datasets. Across the two available databases, we show that in terms of average target recognition accuracy and standard deviation values, the proposed STLFL + DRBM method outperforms traditional methods by 33.5, 78.5, 93.5, and 98.5% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition III datasets II and by 71.3, 100, 100, and 100% for 1, 5, 10, and 15 repetitions, respectively, in BCI competition II datasets II and by 67.5 ± 4, 84.2 ± 2.5, 93.5 ± 1, 96.3 ± 1, and 98.4 ± 0.5% for rapid serial visual presentation (RSVP) based dataset in repetitions 1-5. The method has some advantages over the existing variants including its efficiency, robustness with a small number of training samples, and a high ability to create discriminative features between classes.

The Effectiveness of Narrowing the Window size for LD & HD EMG Channels based on Novel Deep Learning Wavelet Scattering Transform Feature Extraction Approach.

The use of the Electromyogram (EMG) signals as a source of control to command externally powered prostheses is often challenged by the signal complexity and non-stationary behavior. Mainly, two factors affect classification accuracy: selecting the optimum feature extraction methods and overlapping segmentation/window size. Nowadays, studies attempt to use deep learning (DL) methods to improve classification accuracy. However, DL models are frequently hampered by their requirements of a vast quantity of training data to attain decent performance and the high computing costs. Therefore, researchers tried to replace the deep learning models with other low computational cost methods like deep wavelet scattering transform (DWST) as a feature extraction technique. In terms of windows size, selecting a larger window size increases the classification accuracy, but at the same time, it increases the processing time, which makes the system unsuitable for real-time applications. Accordingly, researchers attempted to minimise the size of the overlapping windows as much as possible without impacting classification performance. This work suggests to utilise DWST transform to achieve two goals (a) extracting the features from EMG signal with low computational cost. Even though many studies have used DWST approaches to extract features from other biological signals, but not been examined before for EMG signals. (b) study the effect of extracting the features from high-density EMG datasets (HD EMG) and low-density EMG datasets (LD EMG), reducing the analysis window size by up to 32msec with minimal impact on classification performance. The outcomes of the proposed method are compared with other well-known feature extraction algorithms to validate these achievements. The proposed strategy exceeds other methods by more than 25% in accuracy.

Cardinality and Short-Term Memory Concepts based Novel Feature Extraction for Myoelectric Pattern Recognition.

The quality of the extracted traditional hand-crafted Electromyogram (EMG) features has been recently identified in the literature as a limiting factor prohibiting the translation from laboratory to clinical settings. To address this limitation, a shift of focus from traditional feature extraction methods to deep learning models was witnessed, as the latter can learn the best feature representation for the task at hand. However, while deep learning models achieve promising results based on raw EMG data, their clinical implementation is often challenged due to their significantly high computational costs (significantly large number of generated models' parameters and a huge amount of data needed for training). This paper is focused on combining the simplicity and low computational characteristics of traditional feature extraction with the memory concepts from Long Short-Term Memory (LSTM) models to efficiently extract the spatial-temporal dynamics of the EMG signals. The novelty of the proposed method can be summarized in a) the memory concept leveraged from deep learning structures, capturing short-term temporal dependencies of the EMG signals, b) the use of cardinality to generate logical combinations of spatially distinct EMG signals and as a feature extraction method and 3) low computational costs and the enhanced classification performance. The performance of the proposed method is validated using three EMG databases collected with 1) laboratory hardware (9 transradial amputees and 17 intact-limbed), and 2) wearables (22 intact-limed using two wearable consumer armbands). In comparison to several other well-known methods from the literature, the proposed method shows significantly enhanced myoelectric pattern recognition performance, with accuracies reaching up to 99%.

Combined Dynamic Time Warping and Spatiotemporal Attention for Myoelectric Control.

The success of pattern recognition based upper-limb prostheses control is linked to their ability to extract appropriate features from the electromyogram (EMG) signals. Traditional EMG feature extraction (FE) algorithms fail to extract spatial and inter-temporal information from the raw data, as they consider the EMG channels individually across a set of sliding windows with some degree of overlapping. To tackle these limitations, this paper presents a method that considers the spatial information of multi-channel EMG signals by utilising dynamic time warping (DTW). To satisfy temporal considerations, inspired by Long Short-Term Memory (LSTM) neural networks, our algorithm evolves the DTW feature representation across long and short-term components to capture the temporal dynamics of the EMG signal. As such the contribution of this paper is the development of a recursive spatio-temporal FE method, denoted as Recursive Temporal Warping (RTW). To investigate the performance of the proposed method, an offline EMG pattern recognition study with 53 movement classes performed by 10 subjects wearing 8 to 16 EMG channels was considered with the results compared against several conventional as well as deep learning-based models. We show that the use of the RTW can reduce classification errors significantly, paving the way for future real-time implementation.

Recurrent Fusion of Time-Domain Descriptors Improves EMG-based Hand Movement Recognition.

Controlling powered prostheses with myoelectric pattern recognition (PR) provides a natural human-robot interfacing scheme for amputees who lost their limbs. Research in this direction reveals that the challenges prohibiting reliable clinical translation of myoelectric interfaces are mainly driven by the quality of the extracted features. Hence, developing accurate and reliable feature extraction techniques is of vital importance for facilitating clinical implementation of Electromyogram (EMG) PR systems. To overcome this challenge, we proposed a combination of Range Spatial Filtering (RSF) and Recurrent Fusion of Time Domain Descriptors (RFTDD) in order to improve the classifier performance and make the prosthetic hand control more appropriate for clinical applications. RSF is used to increase the number of EMG signals available for feature extraction by focusing on the spatial information between all possible logical combinations of the physical EMG channels. RFTDD is then used to capture the temporal information by applying a recurrent data fusion process on the resulting orientation-based time-domain (TD) features, with a sigmoidal function to limit the features range and overcome the vanishing amplitudes problem. The main advantages of the proposed method include 1) its potential in capturing the temporal-spatial dependencies of the EMG signals, leading to reduced classification errors, and 2) the simplicity with which the features are extracted, as any kind of simple TD features can be adopted with this method. The performance of the proposed RFTDD is then benchmarked across many well-known TD features individually and as sets to prove the power of the RFTDD method on two EMG datasets with a total of 31 subjects. Testing results revealed an approximate reduction of 12% in classification errors across all subjects when using the proposed method against traditional feature extraction methods.Clinical Relevance-Establishing significance and importance of RFTDD, with simple time-domain features, for robust and low-cost clinical applications.

EMG-Based Hand Gesture Classification with Long Short-Term Memory Deep Recurrent Neural Networks.

Electromyogram (EMG) pattern recognition has been utilized with the traditional machine and deep learning architectures as a control strategy for upper-limb prostheses. However, most of these learning architectures, including those in convolutional neural networks, focus the spatial correlations only; but muscle contractions have a strong temporal dependency. Our primary aim in this paper is to investigate the effectiveness of recurrent deep learning networks in EMG classification as they can learn long-term and non-linear dynamics of time series. We used a Long Short-Term Memory (LSTM-based) neural network to perform multiclass classification with six grip gestures at three different force levels (low, medium, and high) generated by nine amputees. Four different feature sets were extracted from the raw signals and fed to LSTM. Moreover, to investigate a generalization of the proposed method, three different training approaches were tested including 1) training the network with feature extracted from one specific force level and testing it with the same force level, 2) training the network with one specific force level and testing it with two remained force levels, and 3) training the network with all of the force levels and testing it with a single force level. Our results show that LSTM-based neural network can provide reliable performance with average classification errors of around 9% across all nine amputees and force levels. We demonstrate the applicability of deep learning for upperlimb prosthesis control.

Spatially Filtered Low-Density EMG and Time-Domain Descriptors Improves Hand Movement Recognition.

Surface Electromyogram (EMG) pattern recognition has long been utilized for controlling multifunctional myoelectric prostheses. In such an application, a number of EMG channels are usually utilized to acquire more information about the underlying activity of the remaining muscles in the amputee stump. However, despite the multichannel nature of this application, the extracted features are usually acquired from each channel individually, without consideration for the interaction between the different muscles recruited to achieve a specific movement. In this paper, we proposed an approach of spatial filtering, denoted as Range Spatial Filtering (RSF), to increase the number of EMG channels available for feature extraction, by considering the range of all possible logical combinations of each n channels. The proposed RSF method is then combined with conventional time-domain (TD) feature extraction, as an extension of the conventional single channel TD features that are heavily considered in this field. We then show how the addition of a new feature, specifically the minimum absolute value of the range of each two windowed EMG signals, can significantly reduce the different patterns misclassification rate achieved by conventional TD features (with and without our RSF method). The performance of the proposed method is verified on EMG data collected from nine transradial amputees (seven traumatic and two congenital), with six grip and finger movements, for three different levels of forces (low, medium, and high). The classification results showed significant reduction in classification error rates compared to other methods (nearly 10% for some individual TD features and 5% for combined TD features, with Bonferroni corrected p-values <; 0.01).

Spatio-Temporal Inertial Measurements Feature Extraction Improves Hand Movement Pattern Recognition without Electromyography.

Recent studies indicate the limited clinical acceptance of myoelectric prostheses, as upper extremity amputees need improved functionality and more intuitive, effective, and coordinated control of their artificial limbs. Rather than exclusively classifying the electromyogram (EMG) signals, it has been shown that inertial measurements (IMs) can form an excellent complementary signal to the EMG signals to improve the prosthetic control robustness. We present an investigation into the possibility of replacing, rather than complementing, the EMG signals with IMs. We hypothesize that the enhancements achieved by the combined use of the EMG and IM signals may not be significantly different from that achieved by the use of Magnetometer (MAG) or Accelerometer (ACC) signals only, when the temporal and spatial information aspects are considered. A large dataset comprising recordings with 20 ablebodied and two amputee participants, executing 40 movements, was collected. A systematic performance comparison across a number of feature extraction methods was carried out to test our hypothesis. Results suggest that, individually, each of the ACC and MMG signals can form an excellent and potentially independent source of control signal for upper-limb prostheses, with an average classification accuracy of $\approx 93$% across all subjects. This study suggests the feasibility of moving from surface EMG to IM signals as a main source for upper-limb prosthetic control in real-life applications.

Navigating features: a topologically informed chart of electromyographic features space.

The success of biological signal pattern recognition depends crucially on the selection of relevant features. Across signal and imaging modalities, a large number of features have been proposed, leading to feature redundancy and the need for optimal feature set identification. A further complication is that, due to the inherent biological variability, even the same classification problem on different datasets can display variations in the respective optimal sets, casting doubts on the generalizability of relevant features. Here, we approach this problem by leveraging topological tools to create charts of features spaces. These charts highlight feature sub-groups that encode similar information (and their respective similarities) allowing for a principled and interpretable choice of features for classification and analysis. Using multiple electromyographic (EMG) datasets as a case study, we use this feature chart to identify functional groups among 58 state-of-the-art EMG features, and to show that they generalize across three different forearm EMG datasets obtained from able-bodied subjects during hand and finger contractions. We find that these groups describe meaningful non-redundant information, succinctly recapitulating information about different regions of feature space. We then recommend representative features from each group based on maximum class separability, robustness and minimum complexity.

Monitoring of nocturnal central sleep apnea in Heart failure patients using noncontact respiratory differences.

Monitoring of respiration patterns allows the early detection of various breathing disorders and may better identify those at risk for adverse acute outcomes in a variety of clinical settings. In this paper, we report on the use of SleepMinder (SM), a bedside non-contact Doppler-based biomotion recording sensor, to monitor remotely the nocturnal respiration patterns of 50 patients with systolic Heart failure (HF) while undergoing a lab based Polysomnography (PSG) test. A new respiration rate (RR) monitoring algorithm was developed based on the collected overnight radar signals. Two schemes of RR scoring were utilized: respiratory rate count (RRC) and instantaneous respiratory rates (IRR). Analysis of SM vs. PSG revealed that the mean/median IRR scored by SM is highly correlated with that scored on the nasal flow/effort signals from the corresponding PSG studies on all patients, with a significant correlation coefficient of 0.98 (average absolute difference of 0.31 breaths/min), and 0.97 (p<;0.01, average absolute difference of 0.38 breaths/min) for the median and mean of RR respectively. Our experimental results also show that the difference between the RR estimations from IRR and RRC schemes can be utilized to identify central sleep apnea (CSA)/Cheyne-Stokes respiration (CSR) sections without additional apnea detection modules. As a result, with a sensitivity and specificity of 71% and 88% respectively, and an accuracy of 86%, our CSA/CSR screener, plugged with our RR estimation, can play an important role in the remote management of HF patients.

A Framework of Temporal-Spatial Descriptors-Based Feature Extraction for Improved Myoelectric Pattern Recognition.

The extraction of the accurate and efficient descriptors of muscular activity plays an important role in tackling the challenging problem of myoelectric control of powered prostheses. In this paper, we present a new feature extraction framework that aims to give an enhanced representation of muscular activities through increasing the amount of information that can be extracted from individual and combined electromyogram (EMG) channels. We propose to use time-domain descriptors (TDDs) in estimating the EMG signal power spectrum characteristics; a step that preserves the computational power required for the construction of spectral features. Subsequently, TDD is used in a process that involves: 1) representing the temporal evolution of the EMG signals by progressively tracking the correlation between the TDD extracted from each analysis time window and a nonlinearly mapped version of it across the same EMG channel and 2) representing the spatial coherence between the different EMG channels, which is achieved by calculating the correlation between the TDD extracted from the differences of all possible combinations of pairs of channels and their nonlinearly mapped versions. The proposed temporal-spatial descriptors (TSDs) are validated on multiple sparse and high-density (HD) EMG data sets collected from a number of intact-limbed and amputees performing a large number of hand and finger movements. Classification results showed significant reductions in the achieved error rates in comparison to other methods, with the improvement of at least 8% on average across all subjects. Additionally, the proposed TSDs achieved significantly well in problems with HD-EMG with average classification errors of <5% across all subjects using windows lengths of 50 ms only.

Development and validation of a novel non-contact monitor of nocturnal respiration for identifying sleep-disordered breathing in patients with heart failure.

AIMS: At least 50% of patients with heart failure (HF) may have sleep-disordered breathing (SDB). Overnight in-hospital polysomnography (PSG) is considered the gold standard for diagnosis, but a lack of access to such testing contributes to under-diagnosis of SDB. Therefore, there is a need for simple and reliable validated methods to aid diagnosis in patients with HF. The aim of this study was to investigate the accuracy of a non-contact type IV screening device, SleepMinderTM (SM), compared with in-hospital PSG for detecting SDB in patients with HF. METHODS AND RESULTS: The study included 75 adult patients with systolic HF and suspected SDB who underwent simultaneous PSG and SM recordings. An algorithm was developed from the SM signals, using digital signal processing and pattern recognition techniques to calculate the SM apnoea-hypopnoea index (AHI). This was then compared with expert-scored PSGAHI . The SM algorithm had 70% sensitivity and 89% specificity for identifying patients with clinically significant SDB (AHI ≥ 15/h). At this threshold, it had a positive likelihood ratio of 6.3 and a negative likelihood ratio of 0.16. The overall accuracy of the SMAHI algorithm was 85.8% as shown by the area under a receiver operator characteristic curve. The mean AHI with SM was 3.8/h (95% confidence interval 0.5-7.1) lower than that with PSG. CONCLUSIONS: The accuracy of the non-contact type IV screening device SM is good for clinically significant SDB in patients with systolic HF and could be considered as a simple first step in the diagnostic pathway.

Selecting the optimal movement subset with different pattern recognition based EMG control algorithms.

Pattern Recognition (PR)-based EMG controllers of multi-functional upper-limb prostheses have been recently deployed on commercial state-of-the-art prostheses, offering intuitive control with the ability to control large number of movements with fast reaction time. Current challenges with such PR systems include the lack of training and deployment protocols that can help optimize the system's performance based on amputees' needs. Selecting the best subset of movements that each individual amputee can perform will help to exclude movements that have poor performance so that a subject-specific training can be achieved. In this paper, we propose to select the best set of movements that each amputee can perform as well as identifying the movements for which the PR system would have the worst performance and, therefore, would require further training. Unlike previous studies in this direction, different feature extraction and classification methods were utilized to examine if the choice of features/classifiers could affect the best movements subset selection. We performed our experiments on EMG signals collected from four transradial amputees with an accuracy > 97.5% on average across all subjects for the selection of best subset of movements.

Myoelectric feature extraction using temporal-spatial descriptors for multifunction prosthetic hand control.

We tackle the challenging problem of myoelectric prosthesis control with an improved feature extraction algorithm. The proposed algorithm correlates a set of spectral moments and their nonlinearly mapped version across the temporal and spatial domains to form accurate descriptors of muscular activity. The main processing step involves the extraction of the Electromyogram (EMG) signal power spectrum characteristics directly from the time-domain for each analysis window, a step to preserve the computational power required for the construction of spectral features. The subsequent analyses involve computing 1) the correlation between the time-domain descriptors extracted from each analysis window and a nonlinearly mapped version of it across the same EMG channel; representing the temporal evolution of the EMG signals, and 2) the correlation between the descriptors extracted from differences of all possible combinations of channels and a nonlinearly mapped version of them, focusing on how the EMG signals from different channels correlates with each other. The proposed Temporal-Spatial Descriptors (TSDs) are validated on EMG data collected from six transradial amputees performing 11 classes of finger movements. Classification results showed significant reductions (at least 8%) in classification error rates compared to other methods.

Improving the Performance Against Force Variation of EMG Controlled Multifunctional Upper-Limb Prostheses for Transradial Amputees.

We investigate the problem of achieving robust control of hand prostheses by the electromyogram (EMG) of transradial amputees in the presence of variable force levels, as these variations can have a substantial impact on the robustness of the control of the prostheses. We also propose a novel set of features that aim at reducing the impact of force level variations on the prosthesis controlled by amputees. These features characterize the EMG activity by means of the orientation between a set of spectral moments descriptors extracted from the EMG signal and a nonlinearly mapped version of it. At the same time, our feature extraction method processes the EMG signals directly from the time-domain to reduce computational cost. The performance of the proposed features is tested on EMG data collected from nine transradial amputees performing six classes of movements each with three force levels. Our results indicate that the proposed features can achieve significant reductions in classification error rates in comparison to other well-known feature extraction methods, achieving improvements of ≈ 6% to 8% in the average classification performance across all subjects and force levels, when training with all forces.

Influence of multiple dynamic factors on the performance of myoelectric pattern recognition.

Hand motion classification using surface Electromyogram (EMG) signals has been widely studied for the control of powered prosthetics in laboratory conditions. However, clinical applicability has been limited, as imposed by factors like electrodes shift, variations in the contraction force levels, forearm rotation angles, change of limb position and many other factors that all affect the EMG pattern recognition performance. While the impact of several of these factors on EMG parameter estimation and pattern recognition has been considered individually in previous studies, a minimum number of experiments were reported to study the influence of multiple dynamic factors. In this paper, we investigate the combined effect of varying forearm rotation angles and contraction force levels on the robustness of EMG pattern recognition, while utilizing different time-and-frequency based feature extraction methods. The EMG pattern recognition system has been validated on a set of 11 subjects (ten intact-limbed and one bilateral transradial amputee) performing six classes of hand motions, each with three different force levels, each at three different forearm rotation angles, with six EMG electrodes plus an accelerometer on the subjects' forearm. Our results suggest that the performance of the learning algorithms can be improved with the Time-Dependent Power Spectrum Descriptors (TD-PSD) utilized in our experiments, with average classification accuracies of up to 90% across all subjects, force levels, and forearm rotation angles.

Correlation analysis of electromyogram signals for multiuser myoelectric interfaces.

An inability to adapt myoelectric interfaces to a novel user's unique style of hand motion, or even to adapt to the motion style of an opposite limb upon which the interface is trained, are important factors inhibiting the practical application of myoelectric interfaces. This is mainly attributed to the individual differences in the exhibited electromyogram (EMG) signals generated by the muscles of different limbs. We propose in this paper a multiuser myoelectric interface which easily adapts to novel users and maintains good movement recognition performance. The main contribution is a framework for implementing style-independent feature transformation by using canonical correlation analysis (CCA) in which different users' data is projected onto a unified-style space. The proposed idea is summarized into three steps: 1) train a myoelectric pattern classifier on the set of style-independent features extracted from multiple users using the proposed CCA-based mapping; 2) create a new set of features describing the movements of a novel user during a quick calibration session; and 3) project the novel user's features onto a lower dimensional unified-style space with features maximally correlated with training data and classify accordingly. The proposed method has been validated on a set of eight intact-limbed subjects, left-and-right handed, performing ten classes of bilateral synchronous fingers movements with four electrodes on each forearm. The method was able to overcome individual differences through the style-independent framework with accuracies of > 83% across multiple users. Testing was also performed on a set of ten intact-limbed and six below-elbow amputee subjects as they performed finger and thumb movements. The proposed framework allowed us to train the classifier on a normal subject's data while subsequently testing it on an amputee's data after calibration with a performance of > 82% on average across all amputees.